Using soft maximin for risk averse multi-objective decision-making

Abstract Balancing multiple competing and conflicting objectives is an essential task for any artificial intelligence tasked with satisfying human values or preferences. Conflict arises both from misalignment between individuals values, but also value systems held by a single human. Starting principle of loss-aversion, we designed set soft maximin function approaches to multi-objective decision-making. Bench-marking these functions in previously-developed environments, found that one new approach particular, ‘split-function exp-log loss aversion’ (SFELLA), learns faster than the state art thresholded alignment objective method Vamplew (Engineering Applications Artificial Intelligenceg 100:104186, 2021) on three four tasks it was tested on, achieved same optimal performance after learning. SFELLA showed relative robustness improvements against changes scale, which may highlight advantage dealing distribution shifts environment dynamics. We further compared reward exponentials (MORE) approach, performs similarly MORE simple previously-described foraging task, modified resource not depleted as agent worked, collected more very little cost incurred terms old resource. Overall, useful avoiding problems sometimes occur reward-responsive while retaining its conservative, loss-averse incentive structure.